Papers with left-to-right transformer model
Mask-Predict: Parallel Decoding of Conditional Masked Language Models (D19-1)
Copied to clipboard
| Challenge: | a masked language model is used to train a model to predict subsets of mangled words . a parallel decoding algorithm can be used to generate translations in a constant number of iterations. |
| Approach: | They propose a model and a parallel decoding algorithm which train a machine to predict any subset of target words . they introduce conditional masked language models (CMLMs) which are trained with a mangled language model objective . |
| Outcome: | The proposed model improves state-of-the-art performance levels for non-autoregressive and parallel decoding models by over 4 BLEU on average. |